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The satisfiability transition in a random 2-SAT formula occurs when the clauses-to- variables ratio 
a approaches a c (2) = 1. However, long range frustration among unfrozen variable nodes builds up 
only when a > or (2) = 4.4588. For the random 3-SAT problem, we find a long-range frustrated 
mean field solution when a > ctn(3) = 4.1897. The long range frustration order parameter R of 
this solution jumps from zero to a finite positive value at <xr(3), while the energy density increases 
only gradually from zero as a function of a. The SAT-UNSAT transition point of this solution is 
lower than the value of a c (3) = 4.267 obtained by the survey propagation algorithm. Two possible 
reasons for this discrepancy are suggested. 

The zero-temperature phase diagram of the ±J Viana-Bray model is also determined, which is 
identical to that of the random 2-SAT problem. The predicted phase transition between a non- 
frustrated and a long-range frustrated spin glass phase might also be observable in real materials 
at a finite temperature. 
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I. INTRODUCTION 

An attempt was made in a recent paper [l| to study the 
vertex cover problem from the viewpoint of long range 
frustration. The vertex cover problem, sometimes also 
referred to as the hard-core gas condensation problem 
in the physics literature, can be mapped to a spin glass 
model with quenched randomness coming from the un- 
derlying random vertex- vertex connection patterns. In 
the present paper, we extend the basic idea and the 
method of Ref. [l| to spin glass systems with an addi- 
tional kind of quenched randomness due to the random 
couplings among interacting vertices. We demonstrate 
our calculations by working on a spin glass model of the 
random if-satisfiability (i-T-SAT) problem. Our results 
will be compared with known algorithmic and analytical 
conclusions. 

The if-SAT is at the root of computational complexity 
• A K-S AT formula involves N Boolean variables and 
M = aN constraints or clauses, each of which is a dis- 
junction of K Boolean variables or their negations. The 
parameter a is called the clauses-to- variables ratio. A K- 
SAT formula is satisfiable (SAT) if there exists at least 
one assignment of the Boolean variables (a solution) such 
that all clauses are satisfied; otherwise it is unsatisfiable 
(UNSAT). For example, the 2-SAT formula 

(xi V x 2 ) A (xi V x 2 ) A (x~i V x 3 ) A (x 2 V x 3 ) 

with N = 3 and M = 4 has a SAT solution of x\ — x 2 — 
X3 = TRUE. Given a KSAT formula, the objective is to 
judge whether it is satisfiable or not, and if it is, to find a 
satisfying solution. The satisfiability of a 2-SAT formula 
can be determined in a run time that scales linearly with 



the number N of Boolean variables However, to de- 
termine the satisfiability of a JC-SAT formula with K > 3 
is NP-complete, with a computation time that scales ex- 
ponentially with TV in the worst case. The reason for this 
qualitative difference in computational complexity is still 
not completely clear. 

When a if- SAT formula of N variables and M clauses 
is chosen randomly from the ensemble of all such for- 
mulas, it was observed 0, H, that its probability p of 
being satisfiable drops from p ss 1 to p w over a small 
range of the parameter a around certain a c (for K = 3, 
«c«4.2|10). Furthermore, although the if- SAT 
problem (K > 3) in general is NP-complete, the satisfi- 
ability of a randomly chosen formula can often be easily 
determined by a heuristic algorithm, provided that its 
parameter a is not at the vicinity of the SAT-UNSAT 
transition point a c . The really difficult instances of ran- 
dom K-SAT (K > 3) are located at the parameter regime 
of a « a c 3. 

The above mentioned threshold phenomena arc rem- 
iniscent of what is usually observed in a physical sys- 
tem around its phase transition or critical point. Con- 
cepts of spin glass physics, such as replica symmetry 
breaking and proliferation of metastable states, were ap- 
plied to the random K-SAT problem in some recent ar- 
ticles H H [H El 0. For the random 3-SAT prob- 
lem, Mezard and co-authors [Til IT2I ] found that in the 
parameter regime of 3.921 < a < 4.267, there exists a 
hard-SAT phase with positive structural entropy (com- 
plexity). A random 3-SAT formula in this phase is sat- 
isfiable by exponentially many Boolean configurations, 
which are clustered into different macroscopic domains. 
There are also exponentially many metastable domains of 
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configurations which satisfy most but not all the clauses 
(a local search algorithm is likely to be trapped in one of 
these metastable configurational domains). This physi- 
cal insight is implemented in an efficient heuristic survey 
propagation algorithm |13| to find a solution for a ran- 
dom 3-SAT formula. 

Trying to understand more deeply the qualitative dif- 
ference between the 2-SAT and the general A-SAT prob- 
lem (A" > 3), in this paper we re-examine the ran- 
dom 2-SAT and 3-SAT problem by focusing on the is- 
sue of long range frustration among unfrozen Boolean 
variables. A long range frustration order parameter R is 
calculated. We find that, while the SAT-UNSAT tran- 
sition occurs at a c (2) = 1 for the random 2-SAT prob- 
lem, long range frustration only gradually builds up when 
a > or(2) = 4.4588, where the non-frustrated mean field 
solution becomes unstable. For the random 3-SAT prob- 
lem, we find a self-consistent long-range frustrated solu- 
tion when a > afi(3) = 4.189724. The long range frus- 
tration order parameter R of this solution jumps from 
zero to a finite positive value at ai?(3), while the energy 
density increases only gradually from zero as a function 
of a. This solution predicts that the SAT-UNSAT tran- 
sition occurs at a = a^(3). However, for the random 3- 
SAT problem, the non-frustrated trivial mean field solu- 
tion is stable even when a > a^(3); and furthermore, in 
the range of 3.921 < a < 4.267 there is a non-frustrated 
replica symmetry breaking solution of zero energy |12| . 
It is likely that the true SAT-UNSAT transition point 
is at a = a c (3) = 4.267. Two possible reasons for this 
discrepancy between our mean field solution and the so- 
lution of Ref. are discussed in this paper. It appears 
that, when the replica symmetry breaking occurs before 
the onset of long range frustration, the method outlined 
in this paper needs to be improved, and competitions 
among different macroscopic states should be considered 
in the mean field theory. More work is certainly needed 
to overcome this discrepancy. 

This paper is organized as follows. We first define the 
random A-SAT problem in Sec. HU The random 2-SAT 
problem and the ±J Viana-Bray spin glass model are 
then studied in Sec. IIIII The method developed in this 
section is then applied to the more difficult random 3- 
SAT problem in Sec. II VI We summarize and discuss our 
main results in Sec. [V] 



of mean c = Ka. The Poisson distribution is defined as 

„— c k 

f(k,c) = —. (1) 

Each function node a represents a clause (constraint); it 
is linked to A randomly chosen variable nodes i\, . . . ,1k- 
Function node a has an energy E a which is either 
(clause satisfied) or 1 (clause violated): 

Ea = \{ 1 -^, (2) 

r=l 

where J l a r is the edge strength between o and i r : J l a T = 
— 1 or 1 depending on whether Boolean variable i r in 
clause a is negated or not. In a given random A-SAT 
formula, J* r is a quenched random variable with value 
equally distributed over ±1. The total energy of the sys- 
tem for each of the 2 N possible spin configurations is 
expressed as 

M 

E[{a i }] = Y l E a . (3) 

a=l 

As we are interested in the ground-state energy land- 
scape of model Eq. ©, only those configurations which 
have the global minimum energy are considered in later 
discussions. 



III. THE RANDOM 2-SATISFIABILITY 
PROBLEM AND THE ±J VIANA-BRAY SPIN 
GLASS MODEL 

We begin with the random 2-SAT problem. Experi- 
ence gained in this section will help us to understand the 
more difficult random 3-SAT problem in the next sec- 
tion. We discuss the random 2-SAT problem first from 
the viewpoint of long range frustration fSec. lIII ~K\ . Then 
we study it in Sec. IIII Bl through the cavity method of 
Mezard and Parisi |14l ITa ] by setting the long range frus- 
tration order parameter to R = 0. The later treatment 
causes an annoying divergence in the population dynam- 
ics, suggesting the necessity of taking into account the 
effect of long range frustration. In Sec. IIII CI we briefly 
discuss the finite connectivity ± J Viana-Bray spin glass 
model. 



II. SPIN GLASS MODEL OF THE RANDOM 
if-SATISFIABILITY PROBLEM 

We follow Ref. [HI and represent a random A-SAT 
formula by a random factor graph of N variable nodes 
and M = aN function nodes. Each variable node i rep- 
resents a Boolean variable, with a spin value Oi = ±1. 
It is connected to k function nodes, where k is a ran- 
dom integer governed by the Poisson distribution f(k,c) 



A. Long range frustration in a single macroscopic 
state 

The ground-state configurations of a random 2-SAT 
formula may be classified into different macroscopic 
states 0| (hereafter, a macroscopic state is simply re- 
ferred to as a state). Two microscopic configurations in 
the same state are mutually reachable by flipping a finite 
number of spins of one configuration and then letting 
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the system relax. According to this definition of states, 
two configurations of the same state may differ from each 
other in the spin values of O(N) variable nodes (this is 
due to the formation of a percolation cluster of unfrozen 
variable nodes discussed later). We focus on a randomly 
chosen macroscopic state (3. In state (3, the spin value 
of a randomly chosen variable node i may be fixed to 
cr, = +1 (positively frozen), or to cr, = —1 (negatively 
frozen), or fluctuate over ±1 (unfrozen). The fraction 
of positively frozen, negatively frozen, and unfrozen vari- 
able nodes in state j3 is, respectively, q + , q_, and c/o, 
with q + + c/_ + qo = 1. Our approach at the present 
stage is essentially replica symmetric because of the fol- 
lowing reasons: (a) although many macroscopic states 
may coexist, our system is assumed to stay always in the 
same state, (b) a state is characterized by just two mean 
field parameters, qo and R (to be defined later). 

Since the spin values of the unfrozen variable nodes 
fluctuate among different configurations of state /3, the 
possibility of correlation among these fluctuations must 
be taken into account. Consider an unfrozen variable 
node i. If its spin value is externally fixed to a prescribed 
value Oi = a* (cr* = ±1 with equal probability), how 
many other unfrozen variable nodes must eventually fix 
their spins as a consequence? 

For a random factor graph with size N — > oo, the to- 
tal number s of affected variable nodes may scale linearly 
with N and therefore also approaches infinity. If this hap- 
pens, cr* is referred to as a canalizing value for variable 
node i, and variable node i is referred to as type-I un- 
frozen with respect to spin value cr*. This happens with 
probability R, with R being our long range frustration 
order parameter. The total number of affected variable 
nodes may also be finite, s ~ O(l). In this case, vari- 
able node % is type-II unfrozen with respect to spin value 
cr*. We emphasize that, if an unfrozen variable node i is 
type-I unfrozen with respect to spin value say Oi = +1, 
it is not necessarily also type-I unfrozen with respect to 
spin value cr^ = —1; in other words, the fact that cr* is a 
canalyzing value for variable node i does not mean that 
—a* is also a canalyzing value for the same node. In the 
remaining part of this article, to simplify the notation, 
we will frequently use phrases such as "variable node i is 
type-I unfrozen" and "a type-II unfrozen variable node 
i" . It should be understood that, whenever we mention 
that a variable node i is type-I or type-II unfrozen, we 
mean that it is type-I or type-II unfrozen with respect to 
a prescribed spin value a* . 

Based on the bond percolation theory on random 
graphs llq , we know that the percolation clusters evoked 
by two type-I unfrozen variable nodes have a nonzero 
intersection of size proportional to N. Therefore, the 
spin values of all typc-I unfrozen variable nodes must be 
strongly correlated. If we randomly choose two type-I 
unfrozen variable nodes i and j, then with probability 
one half cr* and a* can not be realized simultaneously in 
any configuration of state (3: if c^ = cr*, then aj must 
take the value o~j = — cr*; if <7j = cr*, then cr; must take 



the value cr^ = —cr* . The probability one half comes from 
the fact that both a* and a* are random variables equally 
distributed over ±1 (due to the quenched randomness in 
the edge strengths, as will become clear later). On the 
other hand, two randomly chosen type-II unfrozen vari- 
able nodes are mutually independent, since each of them 
can only influence the spin values of s ~ 0(1) other vari- 
able nodes while the shortest path length between them 
scales as IniV 16]. For later convenience, we denote 
F (s) as the probability that a randomly chosen unfrozen 
variable node i will eventually fix a total number s of 
other unfrozen variable nodes when it is flipped to a pre- 
scribed value oi — cr*, where s is under the constraint 
that s ~ O(l) and therefore limAr^oo s/N = 0. 

We follow the cavity field approach to calculate the 
parameters e/ + , g_, and c/o. We first generate a random 
factor graph G of TV variable nodes and (on average) aN 
function nodes. Then we create a new variable node i 
and connect it to G by a set Vi of new function nodes, 
the size k of this set following the Poisson distribution 
f(k, 2a). Each of these newly constructed function nodes 
is connected to a randomly chosen variable node of G at 
its other end. The enlarged factor graph is denoted as 
G' . It has N + 1 variable nodes and on average aN + 2a 
function nodes. 

We look at the local environment of a newly added 
function node a, which is connected to the new variable 
node i and a randomly chosen variable node j of factor 
graph G. Let us denote cr* = J^. In factor graph G 
variable node j may be frozen to the spin value cr* or to 
— cr*, it may also be unfrozen. Now we ask the question: 
Can function node a be satisfied by variable node j alone? 
There are the following possibilities: 

(i) . Node j is frozen in factor graph G to cr, = cr*. This 

happens with probability (c/ + + c/_)/2 = (1 — qo)/2. 
In this situation, function node a is satisfied. 

(ii) . Node j is unfrozen and cr* is not a canalizing value 

of node j (type-II unfrozen). This happens with 
probability c/o(l — R). In this situation, function 
node a is also satisfiable by flipping aj to the value 

3 ' 

(iii) . Node j is unfrozen and cr* is a canalizing value of 

node j (type-I unfrozen) . This happens with proba- 
bility qaR. In this situation, function node a is also 
satisfiable by flipping Uj to the value cr*. However, 
this spin flip will evoke a percolation cluster (with 
size scaling linearly with N) of unfrozen variable 
nodes, namely, the spin values of these unfrozen 
variable nodes all have to be fixed. 

(iv) . Node j is frozen to Oj = — cr*. This happens with 

probability (q + + q~)/2 = (1 — qo)/2. In this sit- 
uation, function node a is unsatisfiable by node j. 
To satisfy a, variable node i must be fixed to the 
value o"i = J*. 
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In the set Vi, a number k^ u of function nodes, each con- 
nected to variable node i by a positive bond, are facing 
the local environment described by the above-mentioned 
situation (iii). The random integer kf H is governed by 
the Poisson distribution /(fc^,Ai), where Ai = aq^R. 
When ai is set to ai = — 1, these function nodes are not 
satisfied by node i. To satisfy them, the k^~ u unfrozen 
variable nodes attached to the other ends of these func- 
tion nodes must all be flipped to the appropriate spin 
values. However, since each such spin flips will lead to 
a percolation cluster of size approaching infinity, these 
function nodes may not all be simultaneously satisfiable 
due to the possibility of long range frustration among 
the type-I unfrozen variable nodes. These kf u type-I un- 
frozen variable nodes can be divided into two subgroups 
(say Sa and Sb)', variable nodes of the same subgroup 
can take their canalyzing spin values simultaneously in 
state /3, while variable nodes from different subgroups will 
never take their corresponding canalyzing spin values si- 
multaneously in state (3. Because all these kf u variable 
nodes are unfrozen, their spin values for sure will fluctu- 
ate among different miscroscopic configurations. There- 
fore, in some microscopic configurations, variable nodes 
in Sa will take their canalyzing spin values; and in some 
other microscopic configruations, variable nodes in Sb 
will take their canalyzing spin values. When the spin 
value of variable node i is set to ai = — 1, the probability 
Pf(n) that at least n function nodes will be unsatisfied 
due to this type of long range frustration can be expressed 
as 

oo 

F f (n) = /(2n, Ai) C 2 "„ 2- 2 "+ £ f(n', Ai) Cft 2 W , 

ra'=2n+l 

(4) 

where C™, = n'!/(n!(n' — n)\) is the binomial coefficient. 
The first term on the right hand side of Eq. @ corre- 
sponds to the case that the above-mentioned two sub- 
groups have the same size n (kf H — 2n); the second 
term corresponds to the case that these two subgroups 
are not equal in size, one containing n variable nodes 
and the other containing more than n variable nodes 
(k+>2n + l). 

In the set Vi of function nodes, kf v of them are con- 
nected to i by a positive bond and are facing the local 
environment (iv). The random value kf v is governed by 
the Poisson distribution f{kf v , A2), with A2 = a(l — qo)/2. 
These function nodes are definitely unsatisfied when <7i = 
— 1. Therefore, the probability P v (m_) that m_ function 
nodes will be unsatisfied when Oj = — 1 is 

m_ 

P v (m_) = f(n, A 2 ) P £ (m_ - n) . (5) 

n=0 

When <Ji is set to ai = +1, m + function nodes in the 
set Vi may be unsatisfied. It is easy to verify that m+ 
follows the same probability distribution as to_. Since 
the probability distributions for m + and m_ are known, 
the probability qo(i) for variable node i to be unfrozen 



in the corresponding state /?' of the enlarged graph G 
can be calculated. In the large N limit, we assume the 
following convergence condition that 

lim q (i) = q . (6) 

N—>oo 

With this assumption, we can therefore write down the 
following self-consistent equation for the parameter q$\ 

00 00 

*=E E P v(m + ) P v (m_) SZ- , (7) 

m+=0 m_ —0 

where 6 is the Kronecker symbol. It is easy to verify that 
1+ = q- = (1 - <7o)/2. 

We make an important remark here. At the present 
stage of the mean field theory, we focus only on one state 
(3 of factor graph G, and assume that Eq. 10 holds in this 
state (3. When the system has more than one state, there 
may be competitions among these different states. As an 
example of such competitions, consider two states (3\ and 
(3 2 of factor graph G. We assume (3\ is a ground state 
and f3 2 is a ground state or metastable state, therefore 
Ei < E 2 , where E\ (E-i) is the energy of factor graph 
G in state (3\ (/?2)- When the new variable node i is 
added, n\ and 712 of the nearest-neighbor function nodes 
of i may be violated in the two corresponding states f3[ 
and P' 2 of factor graph G' . Therefore, G' have energy 
E[ = Ei + ni and E' 2 = E 2 + n 2 , respectively, in state /3[ 
and (3 2 . We see that, although E\ < E 2 , in the enlarged 
system state j3' 2 may have lower energy than state j3[ if 
m — n 2 > E 2 — E\. To correctly predict the ground 
state energy of the enlarged system G", we may need the 
information on all the low energy states of the original 
system G. We hope to return to this point in a future 
work. In the present paper, we stick to one state (3 of 
factor graph G. 

Now we calculate the order parameter R. Suppose in 
state (3 variable node i is unfrozen, and it is type-II un- 
frozen with respect to the spin value, say, ct; = —1. That 
is, flipping the spin value of node i to <7j = — 1 will cause 
the fixation of the spin values of s ~ O(l) other unfrozen 
variable nodes. Then all the function nodes that are con- 
nected to i by a positive bond should not face the above- 
mentioned local environment (iii) but may face the local 
environment (ii). Consider such a function node a, and 
suppose its other end is connected to a type-II unfrozen 
variable node j by a positive bond (the discussion is the 
same if the bond is negative). When the spin value of 
variable node i is set to <7, = — 1, the spin value of node 
j must be flipped to aj = +1 to satisfy function node 
a. On the other hand, variable node j may be connected 
to other function nodes besides node a; and therefore 
when aj is set to aj = +1, some other type-II unfrozen 
variable nodes will be affected, ... (a "chain reaction"). 
Based on this observation, the probability distribution 
F(s) defined earlier in this subsection can be determined 
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FIG. 1: The long range frustration order parameter R (solid 
lines) and the unfrozen probability go (dashed lines) as a func- 
tion of a for the random 2-SAT problem. The inset is an 
amplification of the region of a G [0, 10]. 

by the following iterative equation: 

oo k 

F(s) = /(0, Aa) S° s +^f(k, A 3 ) £ <5 s s 1+ ... +Sfc JJ , 
k=i { Sl } i=i 

(8) 

where A3 = aqo(l — R) is the average number of function 
nodes that are attached to variable node i by a positive 
bond and are facing the above mentioned local environ- 
ment (ii) . Equation © holds true as long as the cluster of 
affected variable nodes forms a tree, i.e., there is no loops. 
This constraint is satisfied in a random factor graph of 
size N — ► 00, since the shortest loop length between two 
variable nodes scales as In N ^(j while the cluster size s 
scales as 0(1). 

Since R = 1 — J2^Lo ^( s )i fr° m Eq. © we realize that 

R = 1 - exp(-A 3 -R) . (9) 

The unfrozen probability qo and the long range frus- 
tration order parameter R, as calculated by Eqs. Q and 
@, are shown in Fig. 2] as a function of the clauses- 
to-variables ratio a. Onset of spin freeze occurs at 
a = a c (2) = 1. This point, as shown later, is also 
the point of SAT-UNSAT transition, in agreement with 
the rigorous mathematical proof |17| . A typical ran- 
dom 2-SAT formula of a < 1 is satisfiable while a typ- 
ical random 2-SAT formula of a > 1 is unsatisfiable. 
However, onset of long range frustration is delayed to 
a = a R (2) = 4.4588. For cv c (2) < a < a R (2) a typical 
random 2-SAT formula, although being unsatisfiable, is 
not long-range frustrated. We know that to determine 
the satisfiability of a 2-SAT formula is computationally 
easy. It is interesting to point out that, long range frus- 
tration is absent at both sides of the SAT-UNSAT tran- 
sition a c (2) for the random 2-SAT problem. 



Based on our calculation that a randomly constructed 
2-SAT formula in the parameter regime of a c (2) < a < 
aft (2) is not long-range frustrated (R = 0), we hope that 
a Boolean assignment that satisfies the largest possible 
number of its clauses might be easily found in practice, 
despite the fact that the MAX-2-SAT problem is in gen- 
eral NP-hard ■ We will check this point by computer 
simulations in a later work. 

When a > a R (2), long range frustrations exist among 
a fraction of the unfrozen variable nodes. Interestingly, 
the point a — or(2) also marks the beginning of diver- 
gence of the population dynamics of the next subsection. 

Now we turn to the ground-state energy density e. The 
energy increase ei due to the addition of variable node i 
is 

00 00 

e\(a) = P v (m+) P v (m_) min(m+,m_) . 

m+=0 m_ —0 

(10) 

After the addition of variable node i, the factor graph 
G' has N + 1 variable nodes and on average aN + Ka 
function nodes (K = 2). Its clauses-to- variables ratio is 
therefore a' = a + (K — l)a/(N + 1) . Suppose the energy 
density e of the system is a function only of a. Then from 
the identity 

(N + l)e(et) = Ne(a)+e 1 , 
we obtain the following differential equation 

e(a) + (K - \)a^- = ei 
da 

in the limit of large N. By solving this differential equa- 
tion, we obtain that 

1 f a 2 K 

e{a) = (K - p(^j J, « Me i(«) d5 > ( n ) 

where K = 2. According to the author's unpublished cal- 
culations, in the vertex cover problem, the energy density 
derived through this way is the exact ground-state en- 
ergy density when there is no long range frustration; how- 
ever, when R > 0, this energy density is lower than the 
actual ground-state energy densit y o btained by Weigt 
and Hartmann by numerical means |l8j (the reason, how- 
ever, is still unknown). We expect the same situation will 
occur here in the random if-SAT problem. 

When long range frustration exists (R > 0), an upper 
bound for the ground-state energy density e can also be 
derived. For the enlarged factor graph G' to have clauses- 
to-variables ratio a. on average [K — l)a function nodes 
must be removed 14]. The energetic contribution of 
these function nodes must also be removed. This contri- 
bution comes from two parts: (i) the energy sum of these 
individual function nodes, AE = (K — l)a[(l — qo)/2] K 
and, (ii) additional energy AE' caused by possible long 
range frustration among these function nodes. Therefore, 

e(a) = e x - AE - AE'. (12) 
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FIG. 2: The energy density lower bound (solid lines) and 
upper bound (dashed lines) for the random 2-SAT and 3-SAT 
problem as a function of a. 

If we set AE' = 0, Eq. I|12f) then gives an upper bound. 

The energy densities calculated based on Eqs. and 
(|12fl are shown in Fig. [21 When a > a c (2), the energy 
density becomes positive and the system is UNSAT. In 
the range of a c (2) < a < a R (2), both Eq. (JHJ and 
(|12[) give the identical results, indicating that when there 
is no long range frustration, these equations are exact. 
When a > Or(2), the energy density given by Eq. i|12|) 
is systematically higher than that given by Eq. (|llfl . In 
this regime of long range frustration, at the moment the 
author is still unable to give an exact ground-state energy 
density expression. We hope to return to this point in a 
future work. 



B. Random 2-satisfiability studied by population 
dynamics 

In this subsection, we study the random 2-SAT prob- 
lem following the zero-temperature cavity method treat- 
ment of Ref. [12] ■ Since in the parameter range of 
< a < afl(2) there is no long range frustration, wc 
expect this treatment to be exact in this range. 

Suppose there are many macroscopic states of global 
minimum energy. Let us randomly choose a function 
node a and follow one of its edges to a variable node 
i. Before the edge (a,i) is connected, variable node i 
may be positively frozen in some macroscopic states, and 
be negatively frozen in some other macroscopic states, 
and be unfrozen in the remaining macroscopic states. A 
positively (negatively) frozen variable node is said to feel 
a positive (negative) cavity field hi, while an unfrozen 
variable node is said to feel a zero cavity field hi 0] . The 
(modified) cavity field J l a hi distribution among all the 
macroscopic states (referred to as the cavity field survey) 



is denoted by P(J^hi), where J l a is the bond strength 
between function node a and variable node i. Besides 
function node a, node i is connected by positive bonds to 
p other function nodes b\ , . . . , b p and by negative bonds 
to n other function nodes c±, . . . , c„. 

Assuming there is no long range frustration among the 
unfrozen variable nodes (R — 0), one can write down 
the following self-consistent equation for the cavity field 
distribution P(J*ft,-) [3: 

P P n 

P(Jihi) oc []lP{Jkh jr )<Ui jr ][[[P(J$h kl )dh kl ]x 

J r=l 1=1 

p n 

r=l 1=1 

exp(- 2 /A£ 1 /2) (13) 

where 6(x) is the Heaviside step function, 9(x) = 1 if x > 
and 6(x) — otherwise; y is a re-weighting parameter; 
and 

p n 

AE, = Y j e{-Jth jr )+Y i ^-Jt l l h kl ) 
r=l 1=1 

p n 

-l^ei-jfrh^-^ei-jgh^i (w) 

r=l {=1 

is twice the energy caused by the function nodes 
b\ , . . . , 6p, c\ , . . . , c n . 

In the limit of y —> +oo, the general form of the cavity 
field distribution is ^| 

f P+(Jihi) , p+ 

{ + J7 _(i)p_(J*/ li ) . po 

In Eq. I|15|) , p+ , p- , and po are the probabilities of occur- 
rence for the three types of cavity field surveys. P+(k) 
is a probability distribution over positive integers k; and 
P-(k) is a probability distribution over negative integers 
k. The parameters r)o(i), rj+(i), and ?7-(«) are three ran- 
dom variables, rjo(i) + + ??-(«) = 1- 
Based on Eqs. l|13f) and l|15|). one finds that 

po = L ^ > ( 16 ) 

m=0 v ; 

p+ = P- = (l-po)/2. (17) 

When a < ap>(2), the probability po satisfies apo < 1. 
Consequently, the population dynamics at y = oo con- 
verges to the replica symmetric fix point of 770 = 1 ; V+ = 
r;_ = 0. In other words, the system has only one macro- 
scopic state. When a > or(2), this replica symmetric 
fixed point is unstable, and if initially the random vari- 
ables T)o(i) are slightly less than unity, the population 
dynamics at y = 00 becomes divergent. This divergence 
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suggests that, at this parameter regime, the assumption 
of absence of long range frustration becomes invalid. 

To get rid of this divergence, one can run population 
dynamics at a finite positive value of y. A conceptu- 
ally better way is to directly take into account the ef- 
fect of long range frustration in the population dynam- 
ics. Unfortunately, at the moment the author is unable 
to achieve this goal. 



C. The ±J Viana— Bray spin glass model 

We now briefly discuss the zero temperature phase di- 
agram of the ±J Viana-Bray model [13. The model is 
characterized by the Hamiltonian 

E[{<n}] =~Y, J iWi - ( 18 ) 

m 

where the summation is over all the edges (ij) of a Pois- 
son random graph of mean vertex degree c; the edge 
strength = ±1, with equal probability; and Oi = ±1 
is the spin value on vertex i. The zero temperature 
phase diagram of this model was first reported in Ref . [2(j 
through the replica approach. It was found that, when 
c < 1, the system is in the paramagnetic phase with no 
frozen vertices, i.e., qo = 1; when c > 1, the system is 
in the spin glass phase, where some vertices are frozen 
to positive or negative spin values, and qo gradually de- 
creases from unity. 

The Hamiltonian Eq. 1|18|1 is similar to that of the ran- 
dom 2-SAT and can be studied by the same method men- 
tioned in subsection llll Al We find that the zero temper- 
ature phase diagram of model l|18l) is identical to Fig. 
of the random 2-SAT. The transition between the para- 
magnetic phase and the spin glass phase occurs at c = 1 , 
confirming the earlier result of Ref. [2(| . (When the long 
range frustration order parameter is set to R — 0, the or- 
der parameter qo has the same expression as Eq. (15) of 
Ref. In the spin glass phase, long range frustration 

among unfrozen vertices only builds up (R > 0) when 
the mean vertex degree c > 4.4588. 

The ±J finite connectivity spin glass model therefore 
has two types of phase transitions as the mean connec- 
tivity is increased. First there is the frozen transition 
to the spin glass phase; and then there is the transition 
to a long-range frustrated spin glass phase. It is more 
experimentally relevant to discuss the phase transitions 
in terms of temperature. It is accepted that the spin 
glass phase will persist over a finite temperature range 
of < T < T g 0. We expect that when the mean 
vertex degree c > 4.4588, the long-range frustrated spin 
glass phase will also persist over a temperature range of 
< T < T R , with T R <T g . It will be of great interest 
to check the validity of this picture by computer simula- 
tions. 

Real world spin glass materials are three-dimensional, 
and their spin-spin interactions may be more complex 



than that given by Eq. (|18|l . It is not known whether 
or not long range frustrations exist among the unfrozen 
vertices of such systems, and if they do exist, how to 
detect them experimentally. It may be useful to re- 
interprete the already documented experimental obser- 
vations in terms of the long range frustration order pa- 
rameter R. 



IV. THE RANDOM 3-SATISFIABILITY 
PROBLEM 

This section focuses on the random 3-SAT problem. 
We study it using the method of Sec. IIII Al and then 
compare the results with those obtained by population 
dynamics. 

The probability of a randomly chosen variable node 
being positively frozen, negatively frozen, or unfrozen in 
a macroscopic state (3 of global minimum energy is de- 
noted respectively as g+, g_, and qo as in the case of the 
random 2-SAT problem. R is the probability that an un- 
frozen variable node is type-I unfrozen with respect to a 
prescribed spin value. The parameters qo, q+, q~, and R 
can be obtained following the same procedure as was out- 
lined in Sec. 1111 Al We first generate a factor graph G of 
N variable nodes and M — aN function nodes; we then 
create a new variable node i and connect this new node i 
to G through the formation of k new function nodes, with 
k following the Poisson distribution /(fc, 3a). The other 
ends of such a function node a are connected to two ran- 
domly chosen variable nodes j and k of the factor graph 
G. For convenience of discussion, we denote a* = J 3 a and 
°fc = i ano - m the remaining part of this section, when 
we mention that a variable node j is type-I or type-II 
unfrozen, we mean that it is type-I or type-II unfrozen 
with respect to the spin value a* . In state (3, the function 
node a faces one of the following local environments: 

(i) . Node j is frozen in factor graph G to Uj = a* or node 

k is frozen to — o\. This happens with proba- 
bility (3 — 2<7o — ?o)/4. In this situation, function 
node a is satisfied by j or k (or both). 

(ii) . Both node j and node k are type-II unfrozen. This 

happens with probability 5q(1 — R) 2 . In this situa- 
tion, function node a is satisfiable by j and k. 

(iii) . One of the two variable nodes, say j, is type-II un- 

frozen while the other, k, is type-I unfrozen. This 
happens with probability 2q^R(l — R). In this situ- 
ation, node a is satisfiable by j . (It is also satisfiable 
by node k through setting = a* k , but this spin 
flipping will perturb the spin values of O(N) other 
unfrozen variable nodes.) 

(iv) . Both node j and node k are type-I unfrozen, but 

they can not take the spin value Uj = a* and <Jk = 
simultaneously in state (3. This happens with 
probability q^R 2 /2. In this situation, node a is 
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definitely satisfied by one of nodes j and k (in some 
configurations of /3, node j satisfies a while ctj. = 
— Cfc, and in the remaining configurations node k 
satisfies a while <7j = — cr|). 

(v) Both node j and node /c are type-I unfrozen and they 

can take the spin value Uj — a* and Ok — o], simul- 
taneously. This happens with probability q^R 2 /2. 
In this situation, node a is also satisfiable by j and 
k. However, to satisfy a by node j or k will cause 
an extensive perturbation to the spin configuration, 
whereby the spin values of O(N) other unfrozen 
variable nodes will eventually be fixed. 

(vi) . One of the variable nodes, say j, is type-I unfrozen 

while the other is frozen to cr^ = — crjj. This hap- 
pens with probability 2qoq-R. In this situation, 
node a is satisfiable by node j, with the same con- 
sequence as in situation (v). 

(vii) . One of the variable nodes, say j, is type-II un- 

frozen while the other is frozen to = — <J%. This 
happens with probability 2gog-(l — R)- In this sit- 
uation, node a is satisfiable by node j. 

(viii) . Node j is frozen to Cj = —a* and node k is frozen 
to CTfe = — <7j£. This happens with probability q 2 _. 
In this situation, node a is satisfiable only if the 
spin value of variable node i is set to <t^ = J*. 

For the random 3-SAT problem, the parameter go is 
also determined self-consistently by Eq. 10, with the only 
difference that Ai = (3/2)a[$R 2 /2 + 2q q_R] and A 2 = 
(3/2)aq 2 _; and the parameter R is determined by Eq. 0, 
with A3 = 3ag g_(l — R). The lower and upper bound 
of the global minimum energy density can be obtained 
through Eqs. JHJ and {BJl, with K = 3. The values 
of the long range frustration order parameter R and the 
fraction of unfrozen variable nodes qo are shown in Fig. 
as a function of a, and the energy density lower and upper 
bounds are shown in Fig. |2| 

When a < a R (3) = 4.189724, Eqs. and 10 permit 
only a trivial solution of qo = 1 and R = 0, with zero 
energy density. When a > aij(3), this non- frustrated 
replica symmetric solution is still locally stable. However, 
at this parameter range, another stable self-consistent 
solution appears. This nontrivial solution is long-range 
frustrated, with its order parameter R jumps from zero 
to R = 0.2605 at a = or(3), accompanied by a drop of 
qo from qo = 1 to qo = 0.5270. Interestingly, although 
there are jumps for the order parameters R and qo at 
an(3), the energy density of this solution increases only 
gradually from zero as a function of a (see Fig- EJ) - This 
is an improvement over a previous mean field solution. If 
long range frustration was not taken into account, Eq. l(7)l 
predicts a nontrivial solution when a > 4.667, which, 
however, has an unphysical negative energy density as 
long as a < 5.18 11,111. 

The present mean field solution predicts that, in the 
parameter range of a > a^(3) a random 3-SAT formula 




FIG. 3: Long range frustration index R (circles) and fraction 
of unfrozen variables go (squares) as a function of clause-to- 
variable ratio a for the random 3-SAT problem. 



will be long-range frustrated and unsatisfiable. This 
predicted SAT-UNSAT transition point is located be- 
tween the rigorously known lower and upper bounds • 
However, through the zero temperature cavity method 
and population dynamics, it is found in Ref. [12j that, 
when 3.921 < a < 4.267 the random 3-SAT problem 
has another non-frustrated replica symmetry breaking so- 
lution with zero energy and positive structural entropy. 
The structural entropy of this non-frustrated solution be- 
comes negative when a > 4.267, and this point is re- 
garded as the SAT-UNSAT transition point of the ran- 
dom 3-SAT problem. Therefore, there is a clear discrep- 
ancy between the present work and the result of Ref. |12| . 
Two reasons are conceivable for this discrepancy: 

First, the treatment in this paper is essentially replica 
symmetric, with only two order parameters go and -R- 
In the random 3-SAT problem, it is clear that replica 
symmetry breaking occurs before the onset of long range 
frustration. This is different from what happens in the 
random 2-SAT problem of Sec. II I II and the vertex cover 
problem of Ref. lj. In those two systems, replica sym- 
metry breaking occurs concomitantly with the onset of 
long range frustration. When replica symmetry is broken 
and there are many (macroscopic) states of global min- 
imum energy, first, one needs more order parameters to 
characterize the system in the non-frustrated phase; and 
second, in the long-range frustrated phase, the group of 
type-I unfrozen variable nodes of one state as well as their 
spin value correlations may be quite different from those 
of another state. These two points make calculation quite 
difficult. It is hoped that the present work will stimulate 
future efforts on this line. 

Second, the long-range frustrated mean field solution 
does not evolve from the instability of the non-frustrated 
solution. This is clear from the fact that, at a = a^j(3), 
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the long range frustration order parameter jumps to a 
positive value rather than gradually increases from zero. 
This is qualitatively different from what happens in the 
random 2-SAT and the vertex cover problem, where long 
range frustration is caused by a propagation and percola- 
tion of frustration. Maybe the present frustrated solution 
correspond to a metastable macroscopic state rather than 
a state of global minimum energy. We believe that long 
range frustration will also exist in the ground-states of 
the random 3-SAT problem when a is larger than some 
threshold value. If this onset of long range frustration is 
caused be a percolation of frustration as assumed in this 
paper, the order parameter R should gradually increases 
from zero. 

The author has also performed a stability analysis of 
the replica symmetry breaking solution of Ref . |l J ) , to in- 
vestigate whether or not long range frustration will occur 
gradually when a > 4.267. His preliminary simulation 
results indicate that this solution is also locally stable at 
a ~ 4.267. 



V. CONCLUSION AND DISCUSSION 

In a random iC-SAT problem, some of the variable 
nodes are unfrozen, their spin values change in differ- 
ent microscopic configurations of global minimum energy. 
However, these fluctuations of spin values among differ- 
ent unfrozen variable nodes may be highly correlated. In 
this paper, we have investigated the possibility of such 
kind of correlations based on the physical picture of per- 
colation transition. For the random 2-SAT problem, we 
found that long range frustration among unfrozen vari- 
able nodes gradually builds up as the clauses-to- variables 
ratio a exceeds afl(2) = 4.4588. This value is larger 
than the SAT-UNSAT transition point of a c {2) = 1. For 
the random 3-SAT problem, we found a long-range frus- 
trated mean field solution when a > Or(3) = 4.1897. 
The energy density of this solution increases gradually 
from zero as a function of a, while there is a jump in the 
long range frustration order parameter R at an(3). The 
SAT-UNSAT transition point of this nontrivial solution is 
lower than the value of a c (3) = 4.267 as predicted by the 
work of Refs. [Til fl^L ll3T |. Two possible reasons for this 
discrepancy were mentioned. This discrepancy indicates 
that, when replica symmetry breaking occurs before the 
onset of long range frustration, the present mean field 
theory needs to be improved. We hope the present work 
will stimulate further efforts to overcome this difficulty. 

The phase diagram of the finite connectivity ±J 
Viana-Bray spin glass model was also discussed. We 
suggested the possibility of a phase transition from a 



non-frustrated spin glass phase to a long-range frustrated 
spin glass phase as a finite temperature. This prediction 
might be checked by numerical simulations. 

The method used in the present work may also be 
applicable to other finite connectivity spin glass models 
yjfl . We hope that an appropriate combination of long 
range frustration and the cavity method at the first order 
replica symmetry breaking level |14l Il5| will help to ad- 
vance our understanding of spin glass statistical physics. 

There exists a rigorously polynomial algorithm to solve 
the 2-SAT decision problem; but the 3-SAT decision 
problem is NP-complete. The proliferation of macro- 
scopic domains in the solution spaces of the 3-SAT prob- 
lem and its absence in the 2-SAT problem is unlikely 
to be the reason for this qualitative difference in com- 
putational complexity, since there exist class P (poly- 
nomial) combinatorial optimization problems which also 
have exponentially many macroscopic states in their solu- 
tion spaces 21, 22] . In the author's opinion, an instance 
of a combinatorial optimization problem may become in- 
trinsically difficult if long range frustration of the type 
discussed in this paper exists in such a system. Long 
range frustration causes strong correlations among the 
fluctuations of the spin values of a finite fraction of the 
vertices of the system. This means that, if the spin on 
an unfrozen vertex is set to a given value, the spins on 
other O(N) unfrozen vertices should also be set to the 
corresponding values. This spin value fixation process 
may be extremely difficult to achieve even for a global 
algorithm, since the algorithm needs to choose the right 
vertices and the right spin values to fix. 

In the random 2-SAT problem, there is no long range 
frustration at both sides of the SAT-UNSAT transition 
point. In the vertex cover problem, there is an onset 
of long range frustration in the computationally difficult 
case of mean vertex degree c > 2.7183 In the ran- 
dom 3-SAT problem, we know there exists a non-trivial 
solution with a jump of the long range frustration or- 
der parameter R at a#(3). Although an(3) is unlikely 
to be true SAT-UNSAT phase transition point for this 
problem, we expect to observe that, in a correct replica 
symmetry breaking mean field solution long range frus- 
tration will builds up when the clauses-to-variables ratio 
a > a c (3). To construct such a mean field solution is 
still an open problem. 
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